Synthetic image generation is the process of generating two-dimensional (2D) images from digitally created three-dimensional (3D) scenes. The image generation process includes 3D models of objects which are rendered into a 2D image.
To improve the appearance of the 3D models, textures, also known as color maps, are applied to the 3D objects. These textures can have one, two three or more dimensions and are mapped onto the surface of 3D objects using a mapping function. This mapping of textures onto an object is called texture mapping.
The 3D space is mapped to the 2D image using a perspective transformation. This transformation results in close objects being shown as bigger and objects which are farther away being shown as smaller. For closer objects, a higher level of detail is observed on the surface of the object. As the object moves further away from the view point (viewer's perspective), the object gets smaller and the level of detail (LOD) on the object is accordingly reduced. The LOD of the 3D object can be computed via a variety of methods.
As the surface onto which a texture is mapped is reduced in size in the 2D image the original texture map is reduced. This process is called minification. Minification requires that all of the individual colors within the texture (known as texels) are sampled and filtered together to compute the final color value for applying to the 3D object. The area of texels that is sampled from the texture map is based on the projection of the image pixels onto the texture map. For large texture map minification, this filtering would require processing a large number of texels, thus increasing the computation time. This computation time can be significantly reduced by precomputing the filtered results using a number of different methods, including but not limited to mipmapping, summed area tables, and elliptical weighted average.
Mipmapping takes the texture map at its highest level of detail and generates several lower levels of detail texture maps in a series. For example, if the highest level of detail texture map is a color image of 512×512 resolution, mipmapping may generate a series of textures at lower resolutions such as 256×256, 128×128, 64×64, 32×32, 16×16, 8×8, 4×4, 2×2, and 1×1. Each texel in the new image can be calculated by segmenting the higher resolution image into 2×2 pixel segments and computing the average texture values for each of these segments. Other filters to produce the lower resolution images may also be used. For example, a summed area table computes the sum of all texels from the origin of the texture to the current location in the texture. The newly created lower resolution textures represent the original texture in a new image with a different (i.e., reduced) level of detail (LOD). When the 3D object is rendered, a final color value is generated based upon a selection, or selections, from these LOD images.
For the 2D case, the LOD is computed from the size of the projection of the 2D image pixel into the 2D texture space. For the equations below, if pixel coordinates are referred to as x, y then the texture space coordinates are referred to as u, v.
The projection can be represented by the partial derivatives which are estimated from the change in the texture coordinates across a quad. These partial derivatives are also known as the texture coordinate derivative (or gradient). 2D texture mapping has four components du/dx, du/dy, dv/dx, dv/dy. One conventional method for computing LOD is as follows:
                              l          ⁢                                          ⁢          o          ⁢                                          ⁢          d                =                              log            2                    (                      max            ⁢                          {                                                                                                                  (                                                                              ∂                            u                                                                                ∂                            x                                                                          )                                            2                                        +                                                                  (                                                                              ∂                            v                                                                                ∂                            x                                                                          )                                            2                                                                      ,                                                                                                    (                                                                              ∂                            u                                                                                ∂                            y                                                                          )                                            2                                        +                                                                  (                                                                              ∂                            v                                                                                ∂                            y                                                                          )                                            2                                                                                  }                                )                                    Equation        ⁢                                  ⁢                  (          1          )                    
As the texture map and 3D object rotate and the angle changes with respect to the screen coordinates, the above algorithm computes a changing LOD value, though the value should stay constant under rotational change.
The LOD value is used to select one of the LOD images. For objects that are at oblique viewing angles and the projection of the texture is at a large angle to the camera, the above LOD computation will result in selecting over blurred LOD images. Anisotropic filtering is a method of improving the texture filtering by selecting a less blurry LOD image and filtering that level according to the pixel projection into the texture space.
Modern graphics processing units (GPUs) are designed to render 3D scenes (images). A high level diagram of a conventional GPU 100 is shown in FIG. 1. A GPU 100 can include multiple components such as a shader controller and input unit 110, a programmable shader unit 120, a render backend 130 and a memory 140. The programmable shader 120 runs programs that perform rendering algorithms to transform the 3D objects into a 2D image. The programmable shader 120 comprises hundreds of arithmetic logic units (ALUs). Older GPUs used custom hardware logic for the transformation and color computation stages instead of the programmable shader 120.
A shader controller and input unit 110 reads the input for a variety of computer graphics programs that are performed on the programmable shader 120 in a sequence of stages that comprise the graphics pipeline. The shader controller and input unit 110 can contain several fixed function, custom designed hardware stages of the graphics pipeline including, but not limited to: tessellation, triangle setup, and rasterization. These graphics pipeline stages can also be implemented in software on some graphics devices. The shader controller and input unit 110 also controls the execution of the programs on the programmable shader unit 120 by switching between a large number of programs that are waiting to run.
When texture data is accessed via a hardware texture unit or a program that issues a texture instruction running on the programmable shader 120, texture data is read from memory 140 into the texture unit 105 and texels are selected from the texture unit 105 and returned to the programmable shader 120.
Once a final color value is calculated in the programmable shader 120, the result is sent to the render backend unit 130 which tests the visibility of the color and blends the color with existing colors (color and Z operations) in the frame buffer in memory 140 and writes the final result into the memory 140. The color result from the shader can also use Z and color operation units that are implemented in software on programmable devices.
The texture unit 105 computes a LOD value as well as the memory address of the texture before reading it from the memory 140. The programmable shader 120 sends the current pixel position (x,y) and the current texture coordinates for that pixel (u,v) to the texture unit 105. The programmable shader 120 sends a 2×2 group of neighboring pixels to the texture unit 105. Using the 2×2 group, the texture unit 105 can estimate a rate of change of the texture coordinates (called derivatives) with respect to the pixel coordinates using partial differences as followsux=uPixel1−uPixel0;  Equation (2)vx=vPixel1−vPixel0;  Equation (3)uy=uPixel2−uPixel0; and  Equation (4)vy=vPixel2−vPixel0.  Equation (5)These estimates can be arranged as a Jacobian matrix as follows:
                    J        =                              (                                                                                u                    x                                                                                        u                    y                                                                                                                    v                    x                                                                                        v                    y                                                                        )                    .                                    Equation        ⁢                                  ⁢                  (          6          )                    
Using the following equations:A=υx2+υy2;  Equation (7)B=−2(uxvx+uyvy).  Equation (8)C=ux2+uy2; and  Equation (9)F=(uxυy−uyυx)2;  Equation (10)new values aux, auy, avx, avy can be calculated as follows:
                                          au            x                    =                                                    F                ⁡                                  (                                      t                    +                    r                                    )                                                            t                ⁡                                  (                                      s                    +                    t                                    )                                                                    ;                            Equation        ⁢                                  ⁢                  (          11          )                                                              av            x                    =                                    sign              ⁡                              (                Br                )                                      ⁢                                                            F                  ⁡                                      (                                          t                      -                      r                                        )                                                                    t                  ⁡                                      (                                          s                      +                      t                                        )                                                                                      ;                            Equation        ⁢                                  ⁢                  (          12          )                                                              a            ⁢                                                  ⁢                          u              y                                =                                    -                              sign                ⁡                                  (                  Br                  )                                                      ⁢                                                            F                  ⁡                                      (                                          t                      -                      r                                        )                                                                    t                  ⁡                                      (                                          s                      -                      t                                        )                                                                                      ;        and                            Equation        ⁢                                  ⁢                  (          13          )                                                                            av              y                        =                                                            F                  ⁡                                      (                                          t                      +                      r                                        )                                                                    t                  ⁡                                      (                                          s                      -                      t                                        )                                                                                ;                ⁢                                  ⁢                  where          ⁢                      :                          ⁢                                  ⁢                              r            =                          A              -              C                                ,                      s            =                                          A                +                                  C                  ⁢                                                                          ⁢                  and                  ⁢                                                                          ⁢                  t                                            =                                                sign                  ⁡                                      (                    r                    )                                                  ⁢                                                                                                    r                        2                                            +                                              B                        2                                                                              .                                                                                        Equation        ⁢                                  ⁢                  (          14          )                    
The new values aux, auy, avx, avy can be used instead of the texture coordinate partial derivative estimates and substituted into Equation (1) to compute an LOD value. This method improves the LOD determination by ensuring that it is angularly invariant. The method still requires the LOD calculation of Equation (1) and adds a complex solution requiring a large and slow application specific integrated circuit (ASIC) implementation. A new method for determining the LOD value used to select from among a series of texture maps, which is computationally efficient and reduces the size of the corresponding ASIC implementation, is highly desirable.